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FOREWORD 


This  is  the  Final  Biennial  Progress  Report  of  the  Decision 

Sciences  Laboratory.  On  20  June  1966,  it  was  announced  by 

Maj.  Gen.  J.  W.  O'Neill,  Commander  of  ESD ,  (AFSC),  that  the 

laboratory  was  to  be  abolished  because  of  Command-wide  reductions 

in  manpower  authorizations  and  the  comparatively  low  priority  of  the 

program  of  the  laboratory  in  the  Electronic  Systems  Division.  The 

effective  date  of  the  dissolution  is  1  April  1967.  This  Technical  Report 
has  been  reviewed  and  is  approved. 


R(  .  .  JSAF 

Director,  Decision  Sciences  Laboratory 


ABSTRACT 


This  report,  covering  the  period  July  1964  through  June  1966, 
describes  the  exploratory  development  work  performed  by  the 
Decision  Sciences  Laboratory  in  the  areas  of  data  presentation 
and  display,  communications  testing,  problem  solving,  and 
automated  training. 
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HISTORICAL  NOTE 


History 

The  Decision  Sciences  Laboratory,  an  Air  Force  in-house  laboratory 
since  1948,  was  involved  in  planning,  conducting  and  monitoring  basic 
and  applied  research  throughout  its  history.  The  major  emphasis  in  its 
mission  and  program  was  to  look  ahead,  to  carry  on  research  aimed  at 
providing  avenues  to  fill  important  gaps  in  the  knowledge  about  people , 
particularly  in  information  systems  kinds  of  tasks,  intended  always  to 
improve  the  capability  and  effectiveness  of  the  Air  Force. 

In  1948,  assigned  to  Hq  USAF,  it  was  designated  the  Human 
Resources  Research  Laboratory.  In  late  1952  it  was  transferred  to  the 
Air  Research  and  Development  Command  along  with  other  AF  human  factors 
organizations  and  its  name  was  changed  to  Human  Factors  Operations 
Research  Laboratory  and  it  was  tasked,  in  addition  to  its  programs  in 
basic-applied  research  in  human  information  processing,  with  human 
engineering  and  personnel  studies  supporting  AF  operational  commands. 

In  1954,  the  Laboratory  was  re-organized  as  the  Operational  Applications 
Laboratory  and  was  assigned  as  a  laboratory  in  the  Air  Force  Cambridge 
Research  Center.  The  mission  involved  human  factors  research  in 
communications  and  information  processing,  and  human  engineering  and 
systems  analysis  studies. 

In  1961  with  the  re-organization  of  ARDC  and  the  formation  of  the 
AF  Systems  Command  the  laboratory  was  assigned  to  the  Applied  Research 
Division  of  the  Deputy  for  Technology,  AF  Command  and  Control 
Development  Division  with  its  mission  unchanged.  With  the  organization 
of  the  AF  Electronic  Systems  Division  the  Laboratory  was  then  assigned 
at  the  directorate  level  to  the  Deputy  for  Engineering  and  Technology ,  ESD , 
as  the  Decision  Sciences  Laboratory. 

Throughout  its  history  the  laboratory  maintained  a  viable,  productive 
research  program  responsive  to  Air  Force  needs  and  centered  about  "human 
information  processing  behavior.  " 
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ABOUT  THE  DECISION  SCIENCES  LABORATORY 


General  Definition 

The  Decision  Sciences  Laboratory  (DSL)  is  a  behavioral  sciences 
laboratory  especially  concerned  with  the  behavior  of  man  in  complex 
military  environments  involving  the  presentation  and  processing  of 
information.  It  is  a  laboratory  whose  main  function  is  exploring, 
defining,  and  effecting  the  most  efficient  interaction  between  man  and 
machines  in  military  information  systems.  Such  systems  are  sophisticated 
computer  based  electronic  systems  which  gather  and  process  huge 
quantities  of  data,  and  present  information  to  military  commanders  and 
controllers  which  they  then  use  as  aids  in  decision-making.  These 
systems  are  currently  in  use  or  are  anticipated  by  many  United  States 
Air  Force  and  related  Department  of  Defense  agencies. 

DSL  is  an  Air  Force  organization  of  37  individuals, directed  by  an 
Air  Force  officer,  and  its  mission  is  wholly  military.  It  is  unique  among 
military  organizations,  however,  in  that  the  professional  staff  is  85  % 
civilian.  Most  scientists  on  the  staff  are  psychologists,  but  there  are 
also  electronic  and  mechanical  engineers,  mathematicians,  psycho¬ 
linguists,  and  operations  analysts.  The  academic  achievement  of  staff 
members  is  in  almost  every  case  well  beyond  the  Master's  degree. 

Nearly  50%  have  the  doctorate. 

In  the  military  organizational  structure,  DSL  is  a  component  of  the 
Electronic  Systems  Division  (ESD) ,  and  is  directly  responsible  to  the 
Deputy  for  Engineering  and  Technology.  ESD  and  several  related 
facilities  which  contribute  to  its  mission  form  the  complex  of  tech¬ 
nological  activities  at  and  around  L.  G.  Hanscom  Field,  Bedford ,  Mass. 
Roughly  8,000  people  are  employed  by  ESD,  which  is  a  major  branch  of 
the  Air  Force  Systems  Command  (AFSC) ,  one  of  the  major  Air  Force 
commands. 
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Mission 


The  Decision  Sciences  Laboratory  initiates  exploratory  development 
programs.  That  is  to  say,  it  estimates  and  examines  future  operational 
requirements  of  the  Air  Force,  especially  in  the  area  of  information 
processing  systems,  and  determines  from  these  investigations  what 
problems  are  implied  for  the  interaction  of  man  and  machine.  Attempts 
to  fill  critical  gaps  in  knowledge  about  human  performance  and  machine 
design  which  thus  come  to  light  are  made  through  exploratory  development 
projects  under  each  program.  Most  of  the  experimental  work  on  these 
projects  is  conducted  at  the  laboratory,  although  it  is  sometimes  expedient 
to  carry  on  certain  portions  through  contracts  negotiated  with  particular 
industries  or  academic  institutions  where  critical  or  otherwise  unavailable 
facilities  can  be  utilized.  DSL  is  also  responsible  for  designing,  developing, 
procuring,  evaluating,  managing,  and  updating  certain  display  components 
of  Air  Force  information  systems.  Further,  it  provides  engineering  services 
to  all  elements  of  ESD  in  the  areas  of  its  professional  competence,  e  g.  , 
display  characteristics,  human  performance,  and  man-machine  relationship 
problems,  for  both  the  present  and  future  information  system  needs. 

So  there  is  a  continuum  of  scientific  activity  underway  at  DSL. 

Exploratory  development  projects  sometimes  investigate  the  most  abstruse 
and  seemingly  insignificant  of  quantifiable  variables  in  human  behavior. 

But  such  studies  contribute  to  the  growing  body  of  knowledge  which  is  the 

condition  of  activity  at  the  other  end  of  the  continuum - applications,  or 

the  direct  support  of  efforts  in  the  "real  world"  of  computerized  command 
and  control  environments. 

Philosophy 

The  paraphrase  of  the  formal  mission  statement  above  only  barely 
suggests  the  import  of  DSL's  potential  for  electronic  systems  design 
technology. 

In  general.  In  its  present  state  of  development,  the  technology  is 
very  advanced  in  the  production  of  sophisticated  hardware  —  computers, 
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radars,  and  so  on.  The  state-of-the-art  is  advancing  rapidly  as  concerns 

software  -  programing  computers  and  developing  other  means  by  which 

to  use  the  equipment  more  effectively.  But  it  is  embryonic  in  its 
comprehension  of  how  man  can  most  efficiently  function  as  an  integral 

part  of  these  systems  -  in  knowing  how  he  uses  the  information  supplied 

him  to  make  decisions,  in  recognizing  the  effects  of  information  displays 

on  decision-making  behavior  - in  comprehending,  in  short,  how  men 

and  computers  interact. 

In  particular.  In  the  Decision  Sciences  Laboratory,  ESD  possesses  a 
unique  professional  and  technical  potential  in  a  most  critical  part  of  total 
system  design,  namely  in  defining  man's  part  in  the  system.  Electronic 

systems  are  merely  hardware  -  impressive,  maybe,  but  little  more 

without  their  human  users.  Total  electronic  systems  include  people. 

More  accurately,  in  the  complex  of  dove-tailing  events  from  a  radar  sensor's 
detecting  physical  phenomena  through  the  computer's  collecting  and 
analyzing  data  about  them,  to  man's  choosing  one  from  many  possible 
courses  of  action,  man  is  the  most  important  factor.  He  is  most  important 
in  two  ways.  1)  Man  has  the  "last  word.  "  He  is  the  agent  who  must  effect 
the  appropriate  decision  after  sensors  and  computers  have  collected,  pro¬ 
cessed,  and  presented  data  about,  say,  an  attacking  enemy  force.  That 
decision  may  have  very  far  reaching  consequences  indeed.  2)  Very  little 
is  known  about  the  human  element  in  the  pressures  of  a  multi-machine 
information  system  environment,  about  man's  behavior  as  he  operates  such 
complex  equipment,  evaluate  a  complex  military  situation,  assumes  the 
roles  of  commander  and  controller,  and  ultimately  makes  an  involved 
decision.  These  functions,  along  with  the  most  "human"  of  all  man's 

capabilities  -  improvisation,  inventiveness,  imagination,  can  doubtless 

never  be  duplicated  electronically. 

So  we  must  make  the  most  efficient  use  of  this  most  important  part  of 

an  information  system  -  man  -  and  make  it  easier  for  him  to  bring  his 

unique  capabilities  to  bear  on  complicated  problems.  To  do  this,  we  must 
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learn  a  great  deal  more  about  him  than  we  now  know.  What  processes 
underlie  his  very  long-term  memory  for  material  which  would  take 
literally  millions  of  binary  bits  to  describe  electronically?  What  data 
does  he  use  in  selecting  alternative  actions?  How  does  he  weigh  and 
manipulate  these  data?  How  much  data  can  he  be  expected  to  evaluate? 
How  fast?  What  form  should  the  data  take  to  be  most  useful  to  him,  and 
thus  to  the  system  mission?  The  answers  to  these  and  many  similar 
questions  are  vital,  both  in  specific,  concrete  situations  in  the  "real 
world"  and  in  more  general  theoretical  settings,  not  only  so  we  can 
better  use  man  in  military  systems,  but  so  we  can  better  understand 
man  himself.  The  technical  program  at  the  Decision  Sciences  Laboratory 
exists  in  order  to  answer  these  and  similar  questions  about  man  as  the 
modern  military  commander,  controller,  and  information  processor,  and 
about  man  in  his  own  right  so  that  it  can  design  and  develop  machines 
to  serve  him  better. 

Condition  of  Effective  Operation 

Finally,  a  word  should  be  said  about  how  DSL  must  proceed  in  order 
to  hope  to  realize  any  of  the  goals  embodied  in  its  philosophy.  The 
ability  of  DSL  to  make  significant  contributions  to  systems,  either  in  the 
design  or  operational  stages,  is  entirely  dependent  on  its  being  involved 
in  research  which  is  in  the  context  of  total  system  design.  That  is,  in 
order  that  the  findings  of  research  might  have  direct  application,  he 
useful  here  and  now  in  information  systems,  the  laboratory  staff  must  be 
aware  of  the  direction  and  extent  of  progress  in  all  other  related 
technological  areas.  Only  by  this  means  can  it  direct  and  advance  its 
own  research  so  as  to  advance  the  whole  state-of-the-art  purposefully. 
Tri-National  Symposium  on  Information  Presentation 

There  are  many  avenues  to  keeping  in  touch  with  the  advancing  state- 
of-the-art.  Probably  the  most  direct  is  to  be  actively  involved  in  the 
design  and  modification  of  systems  in  the  field.  But  the  exchange  of  views 
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in  a  more  detached  atmosphere  is  equally  vital.  DSL  scientists  participate 
regularly  and  frequently  in  professional  conventions  of  many  sorts,  and 
recently  initiated  a  medium  for  exchanging  information  which  perhaps 
uniquely  meets  its  needs.  This  is  the  Tri-National  Symposium  on 
Information  Presentation,  intended  as  an  annual  meeting  of  behavioral 
scientists  and  professionals  in  related  idsciplines  from  France,  Germany, 
and  the  United  States.  One  of  the  purposes  of  such  an  international 
symposium  is  to  exchange  views  on  the  human-factors  aspects  of  infor¬ 
mation  presentation,  which  is  the  sole  purpose  of  displays  -  to  inform 

their  users.  It  is  aimed  at  understanding  and  coping  with  the  problems 
associated  with  the  respective  roles  of  man  and  the  machines  that  support 
him.  A  general  summary  of  the  symposium  held  in  October  of  1965  follows. 

Presentations  made  by  the  French  participants  focussed  on  the  areas 
of  physiology  and  psychological  stress  and  on  the  application  of  ergonomic 
work  study  methods  in  analysis  of  operator  performance  in  air-defense 
environments.  These  provided  some  novel  insights  into  factors  in  auto¬ 
mated  data  presentation  such  as  effects  on  the  rest-work  cycle  on  alert¬ 
ness  and  the  stability  of  meaning  of  different  levels  of  abstraction  in 
symbolic  coding  as  a  function  of  the  pressure  of  the  job  at  hand.  The 
"re-engineering"  of  display  equipment  for  Naval  Tactical  data  was  also 
described  as  were  a  proposed  performance  evaluation  in  the  STRIDA  air 
defense  system.  Of  major  interest  here  were  the  actual  and  proposed 
methods. 

The  German  delegation  concentrated  in  the  area  of  display  for  the 
control  of  vehicles,  mainly  vertical  take-off  and  landing  aircraft  (VTOL) 
but  including  ships.  Some  really  pioneering  work  was  presented  on 
combinations  of  two  dimensional  display  for  the  three-dimensional  problem 
of  "hovering"  a  VTOL  and  on  "predictive"  displays  to  facilitate  pilot  control 
prior  to  and  during  the  very  extreme  transition  between  vertical  and  hori¬ 
zontal  flight  and  vice  versa. 
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United  States  presentations  were  concentrated  in  the  areas  of  computer 
generated  display.  One  major  area  involved  new  developments  in  computer 
input  and  output  for  direct,  on-line  interaction  between  operational  user 
and  machine  (the  SDC  General  Purpose  Display  System  and  the  Decision 
Sciences  Laboratory  statistical  processing  programs).  Another  major  area 
was  that  of  character  display  -  a  new  system  for  specification  of  require¬ 
ments  for  legibility  and  the  effects  of  TV  scan  resolution  on  character 
quality.  The  U.  S.  also  provided  presentations  on  display  for  vehicular 
control  (The  Rainbow  Landing  System)  and  a  proposed  development  of 
models  of  data  organization  as  a  function  of  decision  task-in-hand  for 
purpose  of  guiding  display  developments. 

The  concensus  of  attendees  has  been  that  several  valuable  purposes 
were  served  by  this  symposium.  There  was  a  very  effective  exchange  of 
information  in  all  directions.  Each  nation  transmitted  and  received  valuable 
data  to  and  from  others  that  would  not  otherwise  have  been  exchanged. 
Representatives  of  the  three  countries  also  engaged  in  many  informal 
discussions,  giving  and  receiving  unpublished  advice  and  benefit  of 
experience.  In  addition,  professional  contacts  in  the  different  areas  of 
specialization  were  established  which  will  continue  to  bear  the  fruit  of 
cross-fertilization  and  international  cooperation  for  years  to  come. 

PROJECTS 

The  projects  which  DSL  has  been  pursuing  for  some  years  are  aimed  at 
fulfilling  certain  present  and  future  operational  needs  of  the  Air  Force.  DSL's 
professional  staff  identify  and  define  the  problems  to  be  solved  by  examining 
the  Technical  Objectives  (TO's)  which  specify  the  entire  range  of  the  Air 
Force's  operational  requirements.  The  laboratory  staff  searches  those  areas 
likely  to  contain  problems  it  is  uniquely  competent  to  tackle  and  considered 
relevant  to  the  mission  of  ESD,  areas  such  as  human  performance,  computer 
and  information  processing  techniques,  intelligence  techniques,  and 
communications.  As  the  outline  of  mission  requirements  states,  the 
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procedure  is  to  analyze  from  the  TO's  implications  for  man-machine 
integration,  and  to  identify  critical  gaps  in  the  knowledge  about  human 
performance  which  come  to  light.  Once  a  need  is  identified,  the 
problem  of  meeting  it  is  attacked  through  an  exploratory  development 
project  which  is  directed  by  one  of  the  laboratory's  division  chiefs. 

This  scientist  is  responsible  for  initiating  the  project,  for  justifying  it 
in  terms  of  Air  Force  needs,  for  specifying  the  objectives  and  the  approach, 
for  subdividing  it  into  a  number  of  workable  tasks,  for  estimating  cost  and 
manpower  requirements,  and  obtaining  ESD  approval  for  its  exploitation. 
Thereafter,  he  is  responsible  for  its  general  administration,  and  for 
documenting  its  progress  in  terms  of  solution  of  the  overall  problem. 

Project  Titles 

From  an  administrative  point  of  view,  DSL's  total  effort  is  currently 
organized  under  and  channeled  toward  completing  three  projects  and  one 
task  which  is  part  of  a  project  managed  by  another  activity.  These  are 
(1)  Project  7  682:  "Man-Computer  Information  Processing,"  (2)  Project 
2806:  "Dynamic  Man-Computer  Interaction, "  (3)  Project  2808:  Psycho¬ 
acoustic  Standards  in  Voice  Communication  System  Evaluation,"  and  (4) 

Task  280111:  "Communication  with  Computers  in  English." 

Nature  of  the  Projects 

Each  project  embraces  an  area  approaching  a  main  branch  of  the  science, 
and  outlines  only  in  the  most  general  terms  the  major  objectives.  Even  the 
tasks  into  which  each  is  divided  are  major  efforts,  and  are  further  broken 
down  into  numerous  studies  and  experiments.  As  one  study  is  completed, 
another  will  be  planned  based  on  the  conclusions  of  the  first.  Occasionally 
a  series  of  experiments  will  suggest  a  new  field  of  inquiry,  and  a  new  project 
will  be  split  off  from  the  old  one.  On  the  other  hand,  as  tasks  progress 
toward  completion,  several  projects  will  be  incorporated  into  one.  Thus  the 
projects  undergo  a  slow  process  of  evolution.  They  are  continuing  in  nature 
and  have  no  specified  completion  dates. 
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The  projects  themselves  do  not  mutually  exclude  one  another.  Much 
of  the  experimental  work  is  directly  relevant  to  two  or  more  projects,  all 
of  which  are  directed  toward  a  quantifiable  description  of  human  behavior. 
Studies  of  human  information  processing,  for  instance,  have  real  meaning 
for  all  of  the  projects  On  the  other  hand,  the  projects  are  not  intended 
to  represent  the  whole  continuum  of  psychological  inquiry.  They  are 
designed,  as  was  previously  suggested,  to  fill  system  needs  which  are 
evaluated  as  being  pressing. 

For  all  these  reasons  —  the  breadth  of  the  projects,  their  evolu¬ 
tionary  nature,  and  the  fact  that  they  do  not  exclude  one  another  — 
it  is  difficult  to  conceive  of  DSL's  effort  in  terms  of  work  completed 
under  a  given  project.  One  should  consider  a  project  as  an  administrative 
necessity  and  convenience. 

BIENNIAL  PROGRESS 

The  projects  listed  in  the  previous  section  have  had  complicated 
histories  during  which  new  tasks  have  been  incorporated  and  original 
ones  terminated.  The  result  is  that  some  work  carried  on  under  one  formal 
project  could  quite  logically  be  justified  as  being  part  of  another.  So 
rather  than  listing  research  under  its  respective  project,  this  section  on 
progress  is  divided  into  research  areas  which  will  be  more  meaningful  to 
readers  not  thoroughly  familiar  with  the  specific  intent  of  each  project. 
Classification  of  DSL's  Effort 

There  are  six  research  areas  into  which  the  whole  spectrum  of  DSL's 
output  can  be  conveniently  divided:  (1)  data  presentation  and  display, 

(2)  learning,  problem  solving  and  decision-making ,  (3)  programed  teaching 
and  automated  training,  (4)  communications,  (5)  applications,  that  is, 
studies  and  consultant  services  in  direct  technical  support  to  planned  and 
existing  systems,  and  (6)  support  services  to  DSL's  automated  laboratory 
facility.  In  this  section  each  area  is  described.  The  description  is 
followed  by  a  bibliography  of  reports  and  papers,  having  meaning  for  the 
area,  completed  during  the  period  from  July  1964  through  June  1966. 
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Bibliographies 

A  few  entries  in  the  bibliography  sections  which  clearly  have 
relevance  to  two  areas  are  included  twice.  Most  entries  have  been 
given  ESD  Technical  Report  numbers  (TR's).  Most  of  those  that  do  not 
are  unpublished  papers  read  at  professional  meetings  or  working  papers 
for  use  inside  the  laboratory.  DDC  numbers  reflect  Defense  Documentation 
Center  control  for  requesting  the  reports. 

Purpose 

The  purpose  of  this  section  is  to  indicate  concretely  the  nature  and 
scope  of  DSL's  effort,  and  to  list  the  results  for  the  record.  It  is  de¬ 
signed  as  a  listing  of  research  completed  since  the  publication  of 
Bibliography  of  Human  Factors  Research  with  Abstracts,  1954  through  19  62  , 
(ESD-TDR-63-603),  and  Decision  Sciences  Laboratory  Biennial  Progress 
Report ,  July  1962  through  June  1964  (ESD-TDR-64-609). 

Data  Presentation  and  Display 

Because  the  effectiveness  and  reliability  of  today's  complex  semi- 
automated  information  systems  depend  so  heavily  on  the  man  involved,  we 
must  describe  and  measure  his  information  processing  behavior  as  precisely 
and  completely  as  possible.  Such  behavior  is  the  composite  of  a  number  of 
closely  related  processes  which  have  been  isolated  from  one  another,  to  as 
great  an  extent  as  possible,  so  that  each  can  be  measured  individually.  One 
such  process  —  the  first  in  the  related  events  which  result  in  a  decision  — 
is  the  way  man  perceives  and  stores  information. 

Objective.  The  purpose  of  studies  in  data  presentation  and  display, 
then,  is  to  quantify  the  variables  which  affect  perception  and  retention  so 
that  it  will  be  possible  to  specify  the  particular  display  features  which  man 
can  best  use  in  any  particular  information  processing  task.  DSL  scientists 
seek  to  be  able  to  specify  what  sizes,  colors,  and  shapes  of  symbols, 
letters,  numbers  and  other  coding  techniques  are  best  for  transmitting  data, 
and  what  kinds  and  amounts  of  data  to  present  so  that  man  can  transform 
them  into  meaningful  information  effectively. 
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Approach  .  Studies  in  perception  are  generally  approached  by  mani¬ 
pulating  quantifiable  display  characteristics,  and  then  measuring  the 
effect  of  such  manipulation  on  man's  ability  to  detect  or  identify  signals. 
But  the  apparently  clear  cut  problem  is  complicated  by  at  least  two  factors. 
First*  it  has  always  been  extremely  difficult  to  say  where  man's  perception 
of  data  stops  and  his  processing  of  them  begins.  Second,  as  displays 
become  increasingly  complex,  the  scope  of  the  research  effort  is  being 
extended  to  include  the  increasingly  complex  effect  of  "display  inter- 
pretability, "  which  here  refers  to  consolidating,  integrating,  and  eval¬ 
uating  information,  and  not  simply  to  detecting  it  and  measuring  the 
legibility  of  the  display  itself. 

Progress.  As  suggested  earlier,  perception,  memory,  decision-making, 
and  problem  solving  are  all  integral  parts  of  information  processing,  and 
achievements  in  one  area  will  unquestionably  benefit  research  in  other 
areas.  However,  the  descriptions  of  research  projects  which  follow  seem 
to  have  relevance  primarily  for  data  presentation  and  display. 

(1)  Experiments  are  complete  in  which  investigators  proposed  to  deter¬ 
mine  subjects'  ability  to  recognize  geometric  forms  of  varying  complexity. 
Both  the  degree  of  complexity  and  time  lag  between  the  first  and  second 
appearance  of  each  figure  was  varied  and  the  effect  measured.  As  a  result, 
a  statement  can  be  made  concerning  the  degree  of  complexity  which  might 
be  effectively  incorporated  into  any  non-verbal  display. 

(2)  An  extensive  series  of  experiments  to  establish  the  range  of  visual 
search  has  been  completed.  These  experiments  were  undertaken  to  deter¬ 
mine  the  manner  in  which  man  scans  displays  and  searches  for  information 
under  a  number  of  varying  conditions.  DSL  has  gleaned  a  great  deal  of 
useful  information  from  these  experiments  about,  for  example,  search 
pattern  shapes,  optimal  numbers  of  elements  in  a  stimulus  matrix,  matrix 
size  as  it  varies  with  man's  exposure  to  it,  and  the  value  of  subdividing 
matrices. 
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(3)  Experiments  are  planned  to  investigate  the  relative  virtues  of 
flat  versus  hemispherical  map  displays.  Tentative  decisions  about 
design  of  the  apparatus  and  variables  to  be  manipulated  have  been 
reached.  If  hemispherical  displays  prove  to  have  distinctive  ad¬ 
vantages,  in  terms  of  target  specification,  there  would  be  an  obvious 
improvement  in  the  effectiveness  of  information  systems  which  display 
large  portions  of  the  earth's  surface  because  present  Lambert  Conformal 
projection  maps  are  necessarily  distorted.  This  research  and  its  results 
are  of  considerable  interest  to  NORAD,  for  display  of  subsatellite 
trajectories. 

(4)  The  users  of  information  systems  are  frequently  faced  with  a 
plethora  of  information  which  they  must  then  sift  through  and  select  what 
they  need.  Data  presentation  and  display  scientists  are  taking  two 
approaches  to  the  problem,  neither  of  which  has  been  adequately  explored. 
First,  experiments  are  being  conducted  in  which  computer  display  texts 
are  presented  in  novel  ways  to  facilitate  reading  efficiency.  Second, 

DSL  is  working  toward  improving  the  reading  efficiency  of  man  himself  by 
using  a  simple  reading  pacer  and  ordinary  texts,  and  experimenting  with 
pacing  the  reader  in  chunks  of  a  paragraph  or  more. 

(5)  The  status  of  research  on  readability  was  assessed.  It  was 
concluded  that  until  investigators  grapple  with  criterion  problems  more 
effectively,  improved  readability  predictors  are  highly  unlikely. 

(6)  A  study  comparing  the  readability  of  electroluminescent  versus 
teletype  displays  for  weather  messages  was  conducted.  The  mostimportant 
results  of  the  study  were  specific  recommendations  for  improved  processing 
of  weather  information,  and  possibly  for  improved  processing  of  information 
for  other  military  systems.  In  addition  to  display  problems  it  is  suggested 
that  some  operators  are  "error  prone"  thus  reducing  system  effectiveness. 
This  factor  should  be  investigated. 

(7)  On  the  applications  end  of  the  research  spectrum,  DSL  scientists 
contribute  to  the  ESD  Technical  Monitor  Program  as  display  consultants  in 
advanced  planning  studies.  DSL  is  represented  in  the  Display  Working 
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Party  of  the  Joint  Technical  Coordinating  Group/Tactical  Air  Control 
System,  which  reviews  programs  in  all  three  military  services.  This 
review  will  consist  of  an  evaluation  and  identification  of  programs  that 
have  application  to  TAC,  to  increase  the  effort  in  those  areas  where 
warranted,  initiate  programs  when  found  necessary,  delete  duplication 
and  combine  programs  where  deemed  desirable.  DSL  scientists  also 
provide  display  consultations  to  users  of  existing  systems  such  as  the 
National  Military  Command  Center  and  to  numerous  System  Project 
Offices.  (See  Applications). 
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Learning,  Problem  Solving  and  Decision-Making 

After  man  detects  displayed  information,  he  must  store  it  and  later 
retrieve  and  process  it  in  order  to  solve  problems  and  make  decisions. 

The  behavioral  sciences  seek  to  enhance  the  processes  of  storage  and 
retrieval  and  consequently  the  operations  involved  in  problem  solving 
and  decision-making.  Many  questions  are  generated  by  such  investi¬ 
gations ,  the  many  answers  to  which  are  only  barely  tentative,  and  in 
attempting  to  arrive  at  valid  conclusions,  human  factors  scientists  are 
forced  to  investigate  extremely  complex  intellectual  processes.  The 
vast  amount  of  scientific  research  literature  focused  on  the  problems  of 
how  information  is  processed  does  not  in  all  cases  apply  specifically  to 
information  system  problems.  Typically,  such  problems  are  compunded 
by  the  pressures  of  an  information  system  environment:  the  need  for  a 
number  of  people  to  confer  and  reach  quick  decisions  on  important  issues. 
Researchers,  therefore,  must  tease  out  what  often  appear  to  be  minor  and 
unimportant  bits  of  knowledge.  These  units  eventually  can  be  integrated 
so  that  explicit  answers  to  the  questions  created  by  these  complex 
situations  can  be  stated. 

Objective  .  Ultimately,  we  must  be  able  to  answer  the  question, 

"what  is  the  most  effective  method  by  which  man  can  process  information 
and  make  decisions  in  a  dynamic,  time-constrained,  multiple-choice 
situation  in  which  the  task  is  vital  and  the  environment  stressful?  More 
particularly,  DSL's  objective  is  investigating  the  particular  aspects  of 
man's  information  processing  behavior  which  should  be  at  an  optimum 
level  even  while  working  under  pressure  in  an  information  system 
environment. 

Approach.  As  in  studies  of  data  presentation  and  display,  the  approach 
is  to  specify  and  quantify  as  precisely  as  possible  the  myriad  of  variables 
which  could  conceivably  affect  man  as  he  stores  information,  solves 
problems,  and  makes  decisions,  especially  under  stress.  DSL  attempts  to 
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simulate  these  complex  conditions  so  that  experimental  studies  can 
be  made  of  the  relevant  variables.  These  variables,  such  as  experi¬ 
ence  levels  of  operators  and  decision  makers,  levels  of  complexity 
of  the  tactical  situation,  various  levels  of  work  load,  amount  of 
accessible  pertinent  information,  and  uncertainty  of  outcome  must  be 
manipulated  to  specify  their  optimum  values  and  ultimately  to  establish 
conditions  for  the  most  effective  dynamic  decision-making. 

Progress.  Some  of  the  very  important  work  accomplished  in  the 
field  of  learning,  decision-making  and  problem-solving  can  be  indicated 
here. 

(1)  Because  the  information  system  environment  is  stressful,  an 
effort  is  currently  underway  to  ascertain  some  of  the  effects  of  speed- 
stress  in  processing  information  from  a  visual  display. 

(2)  The  information  system  operator  is  required  to  recall,  at  random 
intervals,  information  which  he  has  been  storing  while  he  continues  to 
process  new  information.  Special  short-term  memory  experiments  have 
been  designed  which  apply  specifically  to  this  kind  of  activity.  A  study 
completed  in  1964  showed  that  processing  a  query  during  continuous 
short-term  memory  activity  significantly  degrades  recall  more  than  does 
processing  an  intervening  message.  During  the  course  of  this  study,  it 
was  concluded  that  the  rate  and  accuracy  of  acquiring  short-term  memory 
data  could  be  improved  by  using  some  recent  decision  theory  techniques. 

A  study  to  test  this  hypothesis  was  subsequently  completed  which  yielded 
considerably  improved  performance  measures. 

Another  study  demonstrated  that  short-term  memory  may  be  of  greater 
capacity  than  was  previously  realized.  Individuals  inspected  extended 
sequences  of  complex  but  meaningful  visual  configurations,  and  decided 
whether  each  was  occurring  for  the  first  or  second  time  in  a  series.  They 
were  able  to  identify  stimuli  correctly  significantly  greater  than  chance 
even  when  as  many  as  two  hundred  items  intervened  between  the  first  and  second 
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appearances  of  an  item.  The  study  was  extended  to  investigate  the  ability 
to  recognize  pictorial  material  after  delays  of  up  to  one  year  between 
initial  and  subsequent  exposure  to  the  item  Even  with  this  extension 
the  subjects  were  capable  of  correctly  recognizing  pictures  at  a  level 
greater  than  chance. 

(3)  Series  of  experiments  in  the  general  area  of  verbal  learning 
have  been  going  on  for  some  time.  These  have  to  do  with  man's  ability 
to  recall  or  recognize  verbal  series  and  to  retrieve  words  from  them, 
activities  which  are  similar  to  those  required  of  information  system 
personnel.  One  study  measured  the  effect  of  length  of  verbal  series  on 
recall,  and  the  results  suggest  possible  information  retrieval  strategies. 
Another  yielded  specific  information  on  the  influence  of  word-frequency  on 
serial  learning.  Still  another  examined  the  effect  on  recall  of  long  series 
of  words  which  could  be  assigned  to  semantic  categories  such  as  birds  or 
animals.  The  effects  of  varying  such  factors  as  the  number  of  categories 
and  the  number  of  members  in  each  category  were  measured,  and  the  results 
could  have  significant  meaning  for  future  coding  techniques  in  displays. 

A  study  was  undertaken  to  test  the  theory  that  typical  items  in  the  middle 
of  a  series  (where  learning  is  usually  more  difficult)  will  increase  the 
probability  of  such  items  being  recalled.  The  study  did  in  fact  substan¬ 
tiate  the  theory,  and  may  have  implications  for  design  of  future  information 
system  display  materials. 

(4)  One  of  the  unique  features  of  the  information  system  environment 
is  that  decisions  are  often  made  by  teams  of  men  working  together.  Thus  it 

is  vital  that  we  know  reliably  the  extent  to  which  the  whole  range  of  decision¬ 
making  behavior  (from  detecting  information  to  making  a  decision)  may  be 
modified  by  characteristics  of  the  individuals  and  by  social  or  personal 
influences  operating  in  a  group.  The  group  factor  enormously  complicates 
the  problem  of  specifying  optimum  criteria  for  information  system  decision¬ 
making,  and  has  generated  extensive  investigations  of  group  performance 
which  have  been  going  on  for  some  time.  The  experimental  work  is 
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concerned  with  input  processes  in  small  groups:  vigilance,  pattern 
detection,  storage  and  transmission  of  information,  etc.  The  research 
is  inquiring  into  the  reciprocal  influence  of  group  members  under  different 
conditions  of  task  difficulty.  Further  it  is  vital  to  know  reliably  the 
extent  to  which  all  types  of  decision-making  behavior  may  be  modified 
by  the  characteristics  of  the  individuals  and  by  the  social  or  personal 
influences  operating  within  task-oriented  groups  Several  reports  have 
been  completed  and  several  experiments  are  still  in  progress. 

(5)  DSL  recently  evaluated  a  series  of  proposals  and  granted  a 
research  contract,  the  goal  of  which  was  the  specification  of  Air  Force 
weather  forecasters'  decision-making  processes  in  the  field.  The  research 
is  aimed  at  identifying  the  large  number  of  variables  operant  in  weather 
forecasting,  and  may  ultimately  lead  to  defining  the  optimum  variable 
complex  for  decision-making  strategy  for  forecaster  accuracy. 
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Programed  Teaching  and  Automated  Training 

As  information  systems  become  more  complex,  advanced,  and  capable 
of  processing  more  information  in  less  time,  it  becomes  increasingly 

difficult  for  users  -  commander,  controller,  operator,  and  maintenance 

personnel  -  to  exploit  their  potential  and  to  keep  them  operating  efficient¬ 

ly.  In  other  words,  the  very  features  which  make  information  systems 
potentially  more  useful  also  curtail  their  usefulness  by  putting  added 
burdens  on  the  people  who  work  with  them.  Paradoxically,  the  time 
required  to  learn  how  to  perform  the  new  and  complex  intellectual  and 
perceptual  tasks  tends  to  negate  the  value  of  the  improved  hardware. 

Objective.  By  way  of  working  toward  a  solution  of  this  "training 
dilemma,"  DSL  is  carrying  on  vigorous  research  in  programed  teaching 
and  automated  training.  This  is  an  attempt  to  provide  the  kind  of  on-site 
training  materials  necessary  for  establishing  and  maintaining  proficiency 
in  using  sophisticated  electronic  equipment.  But  the  need  for  devices 
which  expedite  the  learning  process  does  not  end  with  computer  operators. 
People  in  many  jobs  requiring  that  a  man  perform  complex  tasks  or  make 
involved  decisions  can  benefit  from  programed  teaching.  Courses,  for 
example,  have  been  developed  for  teaching  managerial  techniques  such 
as  PERT,  described  below,  and  many  of  the  design  principles  evolved 
from  research  for  command  and  control  system  courses  have  been  in¬ 
corporated  in  them. 

Although  the  training  courses  are  different  (some  are  computer  programs 
and  some  are  text  books),  and  although  their  methods  differ  widely,  they 
all  have  much  in  common.  All  are  tutorial  courses  designed  to  minimize 
the  need  for  a  human  teacher.  They  teach,  test  learning,  reteach  if 
necessary,  and  maintain  proficiency  at  a  level  required  by  the  job.  They 
lead  the  learner  by  very  judiciously  selected  increments,  called  "frames," 
to  the  learning  goal.  The  theory  is  that  such  a  process  will  save  time  in 
the  long  run  by  preventing  the  learner  from  forming  misconceptions  which 
would  block  the  ultimate  learning  goal. 
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Specific  objectives  include  (a)  developing  principles  for  designing 
the  hardware  and  software  for  self-instructional  courses,  (b)  developing 
working  models  for  presenting  information  in  programed  instruction,  pro¬ 
gramed  practice,  and  programed  laboratory  sequences  tailored  to  the  many 
different  skills  which  information  system  users  require,  (c)  developing 
automated  techniques  for  design,  construction,  evolution,  and  evaluation 
of  the  programed  materials,  and  (d)  analyzing  problems  associated  with 
implementing  automated  training  in  the  field. 

Approach.  Because  the  need  is  pressing,  research  in  programed 
instruction  and  automated  training  is  being  carried  on  throughout  the 
whole  continuum  of  DSL's  mission  effort,  that  is,  in  exploratory  de¬ 
velopment  projects,  in  special  tests  and  studies,  and  in  direct  support 
of  planned  and  existing  systems  of  many  types. 

More  specifically,  DSL  has  taken  two  related  approaches  toward  de¬ 
veloping  teaching  techniques.  The  first  has  concentrated  on  defining  the 
self-instructional  features  required  in  courses  dealing  with  the  many 
different  duties  at  the  information  system  console.  During  the  studies  of 
typical  users  consoles,  investigators  found  that  only  a  few  simple  additions  4 

would  allow  these  consoles  to  serve  as  part  time  teaching  machines.  This 
led  to  the  second  approach  which  was  to  adapt  certain  aspects  of  operational 
computer  programs  to  the  training  function.  Experimental  models  for  self- 
instructional  computer  programs  have  been  successfully  developed. 

Progress.  The  many  resulting  studies  have  yielded  the  laboratory 
reports  included  in  the  bibliography.  In  turn,  the  experimental  and  con¬ 
ceptual  findings  contained  in  them  have  enabled  scientists  to  design 
better  the  software  and  evaluate  existing  programs,  and  to  turn  out  valuable 
products  based  on  a  considerable  body  of  valid  knowledge. 

One  such  product  is  the  self-instructional  course  in  the  OTC  (Opera¬ 
tional  Test  Capability)  Query  Language  for  the  computer  in  System  47 3L  — 
the  Headquarters  USAF  Control  System.  This  course  is  not  only  designed 
to  teach,  but  to  maintain  and  advance  the  operator's  proficiency  in  the 
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various  ways  of  communicating  with  the  computer.  It  has  been  thoroughly 
tested  and  is  being  successfully  used  in  the  field.  In  addition,  because 
of  its  effectiveness,  it  will  soon  be  used  in  six  other  Air  Force  Commands 
which  use  the  same  communication  process. 

Late  in  1963,  DSL  completed  research  which  suggested  a  programed 
learning  approach  to  training  technicians  in  the  maintenance  of  AN/FST-2 
equipment.  The  AN/FST-2  (Coordinate  Data  Transmitter)  transmits 
quantized  radar  data  free  of  both  clutter  and  undesired  radar  returns  from 
ADC  Long  Range  Radar  Sites  to  SAGE  direction  center,  and  SAGE  depends 
heavily  on  these  data  to  maintain  its  around-the-clock  surveillance.  DSL 
monitored  a  contractor's  production  of  a  text  and  diagrams  to  train  tech¬ 
nicians  to  trouble-shoot  the  system  and  isolate  a  required  replaceable 
part.  The  training  materials  follow  the  Job  Proficiency  Guide  format  which 
the  Air  Force  has  adopted,  and  reflect  the  latest  equipment  configuration  of 
the  AN/FST-2. 

The  PERT  Cost  programed  instruction  text  is  another  valuable  contri¬ 
bution.  DSL  scientists  provided  extensive  consultant  support  during 
t  every  step  in  the  development  of  this  course.  PERT  (Program  Evaluation 

Review  Technique)  is  a  management  tool  widely  used  throughout  the 
Department  of  Defense  and  in  industry,  and  is  designed  to  aid  high  level 
managers  in  evaluating  the  progress  of  a  program  efficiently.  It  is  a 
complicated  technique  which  requires  considerable  time  to  learn  when 
presented  as  a  course.  Since  the  objective  of  such  a  program  was  to 
allow  busy  managers  to  be  freed  for  other  duties,  DSL  some  years  ago 
developed  an  automated  training  course  called  PERTeach  which  permits 
the  manager  to  learn  the  technique  at  his  own  rate.  He  can  spend  as 
much  time  in  learning  at  one  sitting  as  his  other  duties  will  permit,  thus 
saving  him  from  having  to  attend  a  lengthy  lecture  course.  The  PERT  Cost 
instruction  course  builds  on  the  learner's  knowledge  of  the  PERT  system  and 
teaches  him  to  analyze  program  costs.  Both  PERTeach  and  PERT  Cost 
courses  are  bing  used  in  the  Department  of  Defense  PERT  Orientation  and 
Training  Center  and  as  on-the-job  training  aids  throughout  DOD. 
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DSL  has  given  consultation  on  training  problems  to  many  other  agencies 
such  as  the  Air  Force  Ballistic  Missiles  Division,  Industrial  College  of 
the  Armed  Forces,  Air  Training  Command,  Office  of  the  Assistant  Secretary 
of  Defense  (Manpower),  Defense  Communication  Agency,  Federal  Job 
Corps,  Office  of  Naval  Research,  and  various  defense  contractors. 
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Communications 


In  the  most  general  terms,  the  objective  of  any  Air  Force  information 
system  is  to  facilitate  operational  decisions  by  commanders  and  controllers. 
Information  transmitted  through  such  systems,  however,  becomes  useful 
only  when  it  is  presented  to  human  beings  for  interpretation  and  action. 

The  problem  which  generated  studies  in  communications  is  twofold:  first 
Air  Force  operations  depend  to  a  critical  degree  on  information  transmitted 
vocally.  But  command  and  control  personnel  often  require  more  information 
than  they  can  get  quickly  through  direct  talker-listener  channels.  To 
complicate  the  problem,  telephone  and  radio  communication  systems  may 
have  technical  faults  which  impair  intelligibility.  Second,  although  the 
computer  can  be  the  mainstay  of  the  commander,  it  will  be  of  little  use  in 
the  decision-making  process  if  it  yields  or  accepts  information  too  slowly, 
too  quickly,  or  in  a  form  hardly  useable  by  man  So  "communications" 
here  are  of  at  least  two  different  kinds:  communication  between  one  man 
and  another,  and  communication  between  men  and  computers.  The  greater 
part  of  the  discussion  here  centers  on  the  acute  need  to  make  efficient  use 
of  human  speech  and  auditory  perception,  although  there  is  an  equally 
acute  need  to  exploit  the  communicatory  potential  of  the  computer.  Progress 
in  man-computer  communication  is  discussed  later  under  Support  of  the 
Automated  Laboratory. 

Objective.  Research  in  voice  communication  aims  to  identify  and  describe 
the  features  of  auditory  signals  which  operators  and  decision-makers  can  use 
for  rapid,  efficient,  and  unambiguous  communication  in  an  information  system 
environment.  A  related  effort  is  to  improve  methods  for  assessing  radio  and 
telephone  system  performance  and  diagnosing  technical  faults. 

The  objective  of  research  in  man-computer  communication  is  to  develop 
techniques  for  permitting  real-time  interaction  between  commander  and 
computer  components.  DSL  is  working  toward  providing  the  commander 
with  a  computer  language  which  is  flexible  and  efficient,  but  does  not 
require  the  constant  intervention  of  a  professional  programmer.  There  is 
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a  related  effort  to  meet  the  need  of  command  and  control  personnel  to  be 
able  to  update  the  computer's  data  files  rapidly  and  simply  determine 
whether  information  is  being  fed  into  storage  from  one  source  or  a  number 
of  sources  simultaneously. 

Approach.  Research  in  voice  communication  continues  to  explore  the 
effects  of  radio  and  telephone  system  characteristics  on  man's  ability  to 
extract  meaningful  information  from  speech  signals,  and  to  refine  techniques 
for  measuring  intelligibility.  Exploratory  development  research  is  also 
being  conducted  into  determining  the  communicatory  value  of  non-speech 
information  such  as  cues,  independent  of  message  content,  by  which 
listeners  judge  such  factors  as  the  identities  and  emotional  states  of 
talkers.  Identifying  the  psychological  and  behavioral  parameters  that 
differentiate  individual  talkers  and  indicate  emotional  states  has  a  variety 
of  implications  for  the  decision-making  process.  It  could  also  have 
application  to  such  exotic  equipment  as  the  vocal  address  computer  of  the 
future. 

To  achieve  faster  and  more  flexible  communication  between  command 
decision  personnel  and  the  computer,  programing  techniques  are  being  de¬ 
veloped  and  refined  which  will  permit  the  use  of  natural  English  (as  opposed 
to  cryptic  machine  language)  to  retrieve  information  from  machine  storage, 
and  which  will  permit  several  human  communicators  at  one  time  to  query  the 
computer  or  to  update  its  data  files. 

Progress. 

There  have  been  several  efforts  to  minimize  the  problems  created  by 
technical  faults  in  voice  communication  systems,  some  of  which  are 
summarized  below. 

(1)  Technicians  in  the  field  have  for  some  time  been  using  the  Modified 
Rhyme  Test  (MRT)  a  brief,  reliable,  talker-listener  test  of  speech  communi¬ 
cation  systems  —  to  measure  the  intelligibility  of  speech  transmitted  over 
a  system  to  diagnose  technical  faults,  and  indicate  overall  handling  capacity 
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under  operational  conditions.  DSL  scientists  have  been  working  con¬ 
tinuously  to  make  it  even  more  reliable  and  to  expand  its  usefulness. 

Series  of  experiments  were  run  to  establish  its  chief  virtues  over  similar 
tests,  to  determine  the  feasibility  of  combining  speech  intelligibility  and 
speaker  identification  tests,  to  compare  MRT  scores  with  other  intelligibility 
test  scores  when  the  technicians  are  under  various  types  of  stress,  and  to 
determine  the  influences  of  the  test  response  form  and  learning  on  test 
scores. 

(2)  A  speech  Communication  Index  Meter  (SCIM)  has  also  been  in  the 
field  for  some  time.  This  electronic  instrument  measures  the  speech  handling 
capacity  of  a  voice  communication  system  reliably,  quickly,  and  automatically 
by  analyzing  sound  patterns  recorded  on  tapes  and  then  played  over  the 
system.  DSL  has  been  working  to  improve  SCIM's  usefulness  in  a  number  of 
ways:  two  acoustic  couplers  were  developed  so  that  the  microphone  and 
headset  portions  of  the  system  could  be  tested.  The  usefulness  and 
accuracy  of  the  device  for  testing  systems  under  almost  steady-state  noise 
was  demonstrated.  Testing  time  has  been  shortened  from  three  minutes  to  a 
few  seconds. 

(3)  DSL  provided  direct  support  of  system  484N  (Wetwash),  the  Pacific 
Area  Communication  System.  The  laboratory  provided  the  System  Project 
Office  several  input  tapes  which  were  played  over  the  system.  The  re¬ 
corded  results  were  later  analyzed  for  intelligibility  and  a  report  was  sent 
to  the  SPO. 

Efforts  to  identify  the  range  of  physical  and  psychophysical  parameters 
of  the  human  voice  are  important  because  the  results  may  well  increase  the 
voice-carrying  potential  of  existing  communication  systems  and  affect 
design  principles  for  those  of  the  future.  The  design  and  realization  of 
vocal  address  computers  will  also  depend  on  such  knowledge  because  a 
single  electrical  signal  must  be  generated  by  any  voice  articulating  the 
same  word.  Some  recent  work  in  this  field  follows: 

(1)  Several  methodological  studies  were  conducted  to  ascertain  which 
approach  would  likely  be  the  most  productive. 


35 


(2)  Several  versions  of  an  ABX  type  talker  recognition  test  were 
developed.  This  procedure  involves  presentation  of  a  sample  of  speech 
from  speaker  "A",  a  sample  from  speaker  "B"  and  then  a  sample  from 
speaker  "X"  whicn  must  be  identified  by  the  listener  as  either  that  of 
"A"  or  "B".  Data  from  experiments  using  the  final  version  indicates 
that  an  average  of  64.  8%  correct  identification  can  be  achieved  over  a 
wode  range  of  test  conditions. 

DSL's  innovations  in  computer  programing  techniques  and  the  possible 
implications  for  Air  Force  information  systems  are  discussed  in  Support  of 
the  Automated  Laboratory . 
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Applications 

"Applications"  are  at  the  opposite  end  of  the  mission  effort  continuum 
from  exploratory  or  advanced  development  projects,  or  "pure  research,  " 
They  are  the  tangible  results  of  DSL's  labor,  the  work  and  products  put 
to  use,  the  knowledge  derived  from  painstaking  research  put  into  practice 
in  the  field. 

Objective.  The  principle  is  simple:  the  process  is  first  to  theorize, 
then  to  verify  experimentally  and  finally  to  put  into  practice.  But  it  is 
hard  to  imagine  the  complexity  and  amount  of  work  lying  behind,  for 
example,  the  type  of  lettering  used  in  a  certain  display,  let  alone  what 
the  message  should  say  to  best  accomodate  the  human  user.  Any  product 
or  consultation  given  the  Air  Force  is  backed  up  by  many  hours  of  testing, 
re-testing,  experimenting,  collating  and  analyzing  data,  as  well  as  by 
the  special  capabilities  of  the  scientists  who  design  the  product  or  give 
the  opinion.  The  objective  of  the  applications  part  of  the  mission  effort 
is  ultimately  the  objective  of  all  the  work  done  at  DSL;  to  apply  the 
principles  and  techniques  developed  through  research  to  systems  and 
situations  in  the  "real  world.  " 

Approach.  Some  of  the  items  included  here  have  been  mentioned 
earlier  in  other  connections ,  but  all  of  them  involve  direct  support  of 
planned  or  existing  systems.  The  work  includes  such  things  as  insuring 
that  operator  functions  are  properly  allocated,  designating  personnel  and 
training  requirements,  programing  instruction,  and  testing  and  evaluating 
personnel  subsystems.  Some  of  the  work  is  of  a  consultative  nature,  but 
it  includes  some  development  of  "hardware"  (such  as  SCIM  —  Speech 
Communication  System  Index  Meter)  and  "software"  such  as  the  PERT 
COST  course  and  the  Rhyme  Test  described  earlier. 

Progress  Because  of  the  disparate  nature  of  the  work,  a  bibliography 
or  reports  in  the  area  of  applications  must  necessarily  be  small  compared 
to  all  the  work  which  can  be  thought  of  as  being  applied,  A  great  deal  of 
it  is  in  the  form  of  letters ,  informal  conversations ,  and  consultations  with 
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contractors  who  build  the  hardware  and  with  the  epole  who  use  the 
systems  already  in  operation.  To  indicate  the  depth  and  scope  of 
contributions  to  the  "real  world",  a  brief  description  of  some  of  the 
work  is  appropriate  here. 

(1)  DSL  is  giving  a  wide  range  of  support  to  the  National  Military 
Command  Center  (NMCC).  Human  factors  studies  have  been  made  which 
contributed  to  the  designing  and  implementing  of  the  current  configuration; 
scientists  made  a  complete  study  of  the  NMCC  to  determine  user  needs 
which  could  be  assisted  by  DSL;  detailed  descriptions  of  recommended 
action  were  made  in  the  human  factors  and  display  areas;  DSL  personnel 
contributed  to  the  designing ,  specifying,  purchasing,  installing,  and 
testing  of  several  displays  and  associated  controls;  several  maintenance 
and  operators'  manuals  were  written. 

(2)  A  number  of  DSL  scientists  were  members  of  the  Human  Engineering 
Panel  and  Personnel  and  Training  Panels  of  the  AFSC  Personnel  Subsystem 
Working  Group.  Several  reports  were  submitted,  a  number  of  contributions 
were  made  to  MIL-STD-803  series  manuals,  Aerospace  Human  Engineering, 
and  several  other  AFSC  documents  were  rewritten  to  align  them  with  the 
latest  developments  in  human  engineering. 

(3)  DSL  was  represented  on  a  team  of  scientists  responsible  for 
formulating  and  defining  possible  communication  experiments  to  be  con¬ 
ducted  first  in  the  ESD/Mitre  Simulation  Laboratory,  and  subsequently 
aboard  the  future  Manned  Orbital  Laboratory  (MOL).  A  formal  proposal 
was  submitted. 

(4)  DSL  voice  communication  scientists  conducted  speech  intelligibility 
tests  of  Wetwash  A,  the  Pacific  Area  Communications  System,  and  sub¬ 
mitted  a  report  of  findings.  (See  Communications) 

(5)  Consultation  services  were  provided  and  a  report  submitted  to 
ESD's  Computer  Division  on  the  Man-Job-Match  Computer  Model  Study. 

(6)  One  scientist  invented  and  produced  a  slide  rule  on  which  the 
subject  in  an  experiment  can  record  his  "belief  state,  "  a  mathematical 
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statement  reflecting  his  reaction  to  evidence  during  certain  kinds  of 
decision-making  experiments.  The  subject  in  a  multiple  choice 
situation  distributes  a  quota  of  points  over  the  alternatives  in  such  a 
way  as  to  indicate  the  strength  of  his  belief  in  the  correctness  of  each 
choice.  The  slide  rule  further  prevents  him  from  distributing  more  than 
an  allotted  quota  of  points  over  the  alternatives. 

(7)  Human  factors  support  was  given  to  a  working  group  charged 
with  developing  a  proposal  for  a  new  ESD  organization  which  would 
provide  improved  system  design  capability,  and  a  report  describing  the 
structure  and  mission  of  the  proposed  organization  was  prepared. 

(8)  DSL  monitored  the  production  of  automated  training  courses  in 
the  OTC  Query  Language  for  the  Headquarters  USAF  Control  System,  in 
the  maintenance  of  the  AN/FST-2  Coordinate  Data  Transmitter,  and  in 
PERT/Cost.  (See  Programmed  Teaching  and  Automated  Training.  ) 

(9)  DSL  is  continuing  to  work  with  the  Mitre  Corporation  in  a 
project  dealing  with  display  problems  in  ESD  L-Systems. 

(10)  Human  factors  and  display  problems  support  was  afforded  to 
the  492L  System  (USSTRICOM  Command  Center  Improvement  Program), 
the  466L  System  (development  of  display  requirements  for  SIGINT  the  Air 
Transportable  Signal  Intelligence  System),  the  44 1L  System  (The  AN/FPS- 
95  Radar  Display  Subsystem),  Preliminary  Design  Review  Board  for  the 
AN/FPS-85  Radar  System,  the  49 3L  System  (VOCOM)  during  Category  I 
testing,  the  416N  SLBM  (Surface  Launch  Ballistic  Missile)  Program,  and 
the  416M  BUIC  (Back-up  Interceptor  Control)  Program  during  Category  II 
testing. 
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Support  of  the  Automated  Laboratory 

DSL's  PDP-1  digital  computer  is  the  nerve  center  of  what  has  come 
to  be  called  its  "Automated  Laboratory  Facility.  "  The  computer  is  small, 
rapidly  operating,  and  expandible,  and  is  being  used  extensively  as  a 
stimulus  generator  and  data  collector,  processor  and  evaluator.  It 
really  affords  two  major  advantages.  (1)  It  greatly  increases  efficiency 
in  that  it  saves  time,  conserves  manpower,  and  affords  perfect  standard¬ 
ization  of  experimental  conditions.  (2)  It  permits  kinds  of  experimentation 
not  possible  with  conventional  techniques.  Experiments  themselves  can 
be  dynamic,  e.  g.  ,  the  character  of  the  next  stimulus  presentation  can  be 
made  to  be  a  function  of  the  sequence  of  responses  preceding  it. 

The  computer  has  been  greatly  expanded  over  the  last  two  years,  thus 
increasing  its  potential.  With  the  expansion,  though,  increased  complexity 
has  also  been  added.  In  order  to  realize  the  effectiveness  of  such  increased 
potential,  part  of  DSL's  total  effort  is  in  the  form  of  technical  support  to 
task  and  project  scientists 

Objectives.  DSL's  Automated  Laboratory  Division  scientists  undertake 
to  develop  computer  techniques  and  routines  which  will  make  experimentation 
easier  and  more  efficient,  and  facilitate  programing  for  text  preparation, 
editing,  storage,  and  retrieval. 

As  hardware  and  software  become  more  complex,  there  is  an  increasingly 
obvious  need  to  document  existing  and  new  programs.  There  is  a  correspond¬ 
ing  effort  to  write  up  these  holdings  and  advances  in  an  easy  to  understand 
form  and  to  make  them  readily  available  to  the  user.  Such  is  constantly 
being  accomplished. 

Approach.  The  many  faceted  problems  are  approached  by  forseeing  as 
fully  as  possible  the  laboratory's  programing  needs,  and  by  working  as 
closely  as  possible  with  experimenters  in  developing  programs  and  equip¬ 
ment  as  needed. 

Progress.  As  indicated  above,  support  of  the  automated  laboratory 
facilities  takes  several  different  forms,  A  summary  of  some  recent 
accomplishments  follows. 
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(1)  Work  done  on  hardware  used  in  in-house  experiments  and  field 
projects  includes:  several  modifications  of  the  Forced  Choice  Four 
Interval  Stimulus-Response  Facility  in  ESD's  anechoic  chamber;  setting 
up  two  speech  intelligibility  testing  facilities;  the  assembly  and  thorough 
testing  and  adjusting  of  equipment  for  speech  intelligibility  testing  in  the 
field;  setting  up  a  photo  laboratory  which  now  produces  slides,  copy,  and 
good  quality  visual  materials  as  needed;  building  a  self-contained  unit 
which  houses  an  automatic  computer-controlled  camera  for  making  clear 
photographs  of  the  computer  scope-face  up  to  the  rate  of  two  per  second 
and  several  thousand  in  number;  and  allows  for  the  operation  of  various 
auditory,  light,  and  vibrotactile  stimulus  generators. 

(2)  A  concerted  effort  is  nearing  completion  to  document  the  automated 
laboratory  system,  i.e.  ,  to  produce  up-to-date  reference  documents  on  the 
laboratory's  hardware  and  software  system  so  that  new  users  who  are  not 
professional  programers  can  quickly  become  familiar  with  the  computer  and 
the  major  features  of  the  DSL  programing  system. 

(3)  Another  major  effort  to  increase  the  computer's  usefulness  is  the 
development  of  a  matrix  algebra  interpreter  package  (MAP)  and  a  command 
language  for  use  with  it.  This  package  will  allow  a  real  time  interaction 
of  scientist  and  computer  in  formulating  and  solving  classes  of  mathe¬ 
matical  problems  which  are  widely  applicable  in  both  physical  and 
behavioral  sciences,  and  which  are  prohibitively  complex  without  re¬ 
course  to  a  computer.  As  an  adjunct  to  the  algebraic  interpreter,  a 
system  will  be  developed  which  will  decode  alphanumeric  strings 
composed  of  algebraic  operators  and  arbitrarily  labeled  variables,  re¬ 
cording  them  in  appropriate  sequences  of  calls  to  MAP  procedures.  The 
interpreter  will  provide  the  user  scientist  with  real  time  access  to  power¬ 
ful  analytic  tools  without  imposing  demands  on  him.  He  will  have  to  be 
familiar  only  with  a  reasonably  standard  algebraic  notation  and  a  few 
simple  rules  for  exercising  input-output  options.  The  system  should  be 
ready  for  use  very  soon. 
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Relevance  for  Air  Force  Systems 

Although  it  appears  that  the  innovations  in  programing  techniques 
described  here  have  meaning  only  for  laboratory  experimenters,  some  of 
them  evolve  as  products  which  are  directly  applicable  to  information 
systems.  Whenever  a  commander,  controller,  or  operator  faces  a 
computer  which  yields  or  accepts  information  in  a  cryptic  form,  the 
system's  function,  to  facilitate  the  decision-making  process,  breaks 
down.  DSL's  efforts  to  provide  computer  programs  and  communication 
techniques  for  experimenters  do,  in  fact,  have  application  outside  the 
laboratory  environment  New  and  more  efficient  techniques  for  retrieving 
information  from  document  collections  are  currently  being  field  tested  in 
an  operating  environment  with  the  cooperation  of  NASA,  A  statistical 
analysis  of  language  regularities  is  being  run  to  test  an  assumption  about 
the  distributions  of  synonyms  in  text.  Verifying  this  assumption  would 
validate  and  give  insight  into  the  methods  used  in  most  operating  infor¬ 
mation  retrieval  systems,  and  may  lead  to  methods  for  handling  semantics 
in  text.  DSL  plans  in  the  near  future  to  consider  techniques  for 
implementing  self-extending  computer  languages  using  the  computer 
display  scope  and  light-pen.  A  self-extending  language  is  one  to 
which  definitions  of  new  operations  can  be  added,  and  such  a  capability 
would  give  users  an  important  increase  in  flexibility. 
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